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Description 

[0001] The present invention relates to a communica- 
tions method for carrying out communications using im- 
ages for videophones , TV conferences , and video mails, 
or a communications control method forcontrolling com- 
munications using images between communications 
devices, and a device for the same method. 
[0002] As generally known, as a device and method 
for communications using images, there are various 
systems such as videophones, TV conference systems, 
and video mail. The main purpose in use of images in 
all of these systems is to image an appearance of an 
object (for example, a face) and transmit the image to a 
communications destination. 

[0003] However, these systems lack consideration In 
privacy protection. For example, in a case where an ob- 
ject is a user himself/herself, since image data capturing 
the user s face is transmitted to a communications part- 
ner without any processing and is displayed on a display 
of the communications partner, a user who does not 
want the communications partner to see his/her face 
cannot protect his/her privacy, and suffers mental dis- 
tress. 

[0004] Also, when such a system is used at home or 
an office, even in a case where a user does not want to 
transmit images showing a condition inside the home or 
office, images showing the condition are transmitted to- 
gether with the user's appearance, and this is disadvan- 
tageous in terms of protection of privacy and confiden- 
tiality inside the office. 

[0005] Therefore, in order to cope with such a prob- 
lem, Japanese Unexamined Patent Publication No. 
H05-91 407 discloses a technique for protecting domes- 
tic privacy and confidentiality inside an office by elimi- 
nating the background from images shot by a camera 
and transmitting the images. 

[0006] Furthermore, Japanese Unexamined Patent 
Publication No. H06-46414 discloses a technique for 
protecting users' privacy in which the facial portton of a 
person is detected from an image inputted from a cam- 
era and processed so as not to be recognized by others, 
and then transmitted to the other party. 
[0007] Moreover, Japanese Unexamined Patent Pub- 
lication No. 2000-175168 discloses a technique for 
transmitting user's emotions using images while pro- 
tecting privacy by displaying representative images 
such as character images which are optionally variable 
and represent the communications partner's real imag- 
es on display screens of the communications partner. 
[0008] Currently, due to the development of broad- 
band networks, communications using images have be- 
come more commonplace than previously. Accordingly, 
it has become a matter of course to store communica- 
tions data and reuse the data at a later time. Therefore, 
demand for protection of privacy of images which image 
a user himself/herself have increased more than before. 
[0009] Furthennore, it is easily expected from the cur- 



rent situation of electronic mail systems and web sys- 
tems that user demand for convenience and ease in im- 
age data retrieval and management will increase in or- 
der to store and reuse image data. 

5 [0010] On the other hand, as a communications meth- 
od in which a plurality of users at multipoints communi- 
cate using images, a TV conference system exists using 
a centralized operation unit. In this case, it is possible 
that an image to be transmitted is transmitted to two or 

10 more image communications devices from one image 
communications device via a centralized operation unit. 
[0011] In such a TV conference system at multipoints, 
in a case where a technique for protecting privacy is ap- 
plied, for example, by eliminating a background or 

IS processing a facial region for privacy protection, an im- 
age In which a background has been eliminated or a fa- 
cial region has been processed is uniformly transmitted 
to all members participating in a conference. 
[0012] As mentioned above, among systems in which 

20 a plurality of users at multipoints communicate using im- 
ages, in addition to systems to be used exclusively for 
conferences, there is a case where unspecified users 
participate such as in web chats (hereinafter, referred to 
as video chats) and a case where communications are 

25 carried out among a plurality of users at multipoints us- 
ing video mail or mail attached with images. 
[0013] In such a case, demand for controlling image 
disclosure infomnation (that is, information as to whether 
or not an object region is disclosed and whether or not 

30 a background region is disclosed) depending on a com- 
munications partner arises. For example, a case can be 
supposed where an image communications device user 
does not mind revealing his/her face to a specific image 
communications device user, but minds to reveal his/her 

35 face to other users. 

[0014] Also, in TV conference systems for a business 
conference at multipoints, In a case where both in-house 
users and outside users exist, it is considered that im- 
ages are transmitted to outside users after eliminating 

40 a background although the background is allowed to be 
seen by in-house users. 

[0015] However, In the case of the conventional tech- 
nique, when ft is desired to eliminate a background or 
process a facial region for privacy protection, images in 

45 which backgrounds have been eliminated or facial re- 
gions have been processed must be uniformly transmit- 
ted to all members joining a video chat or TV confer- 
ence. This becomes very inconvenient for users. 
[0016] Furthermore, in a case where communications 

50 data including these images is recorded and managed 
as they are, at a later time, it is desired to retrieve back- 
grounds and unprocessed facial regions, and it is re- 
quired to scan the whole image series^ resulting in poor 
efficiency. 

55 [0017] Furthermore, in management of a recorded im- 
age series, since information on disclosure of the data 
of an image series is separately recorded and managed, 
such infomnation may be lost, misplaced, or deleted by 
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mistake. 

[0018] Therefore, an object of the invention is to pro- 
vide a connmunications method in which a function of 
privacy protection is added for communications of im- 
ages, and even when the function of privacy protection 
is added, privacy protection is convenient for users and 
can be easily controlled in reuse of communications da- 
ta and communications among multipoints. 
[0019] The present invention makes it possible to 
communicate an image series including one or more im- 
ages and control information which is related to at least 
one of the image series and Indicates whether or not the 
image is disclosed, wherein the control information is 
related to either or both of a focused region including a 
part or all of an object within an image and a non-fo- 
cused region. 

[0020] The present invention solves the problems In 
the prior art by transmitting and receiving control infor- 
mation together with images. 

[0021] As embodiments of a system according to the 
invention, a first embodiment to carry out one-on-one 
communications between communications devices and 
a second embodiment to carry out multi-point commu- 
nications among three or more communications devices 
by a communications control unit for controlling commu- 
nications among the communications devices are con- 
sidered. 

[0022] The above, and other objects, features and ad- 
vantages of the present invention will become apparent 
from the following description read in conjunction with 
the accompanying drawings, in which like reference nu- 
merals designate the same elements. 

Fig. 1 is a block diagram of a communications de- 
vice in the first embodiment of the invention; 
Fig. 2 is an explanatory view of a meta-data describ- 
ing format of the same; 

Fig. 3 is a drawing showing the relation of an image 
series and meta-data of the same; 
Fig. 4 and Fig. 5 are drawings showing examples 
describing the same meta-data in the MPEG-7 for- 
mat; 

Fig. 6 is a format definition chart of the "Media Time 
Point Type" format of the same data; 
Fig. 7 is an illustration of original input data of the 
same data; 

Figs. 8(a) and (b) are illustrations in which a non- 
focused region (background) of the same data is set 
as a nondisclosure region; 

Figs. 9(a) through (c) are illustrations in which a fo- 
cused region of the same data is set as a nondis- 
closure region; 

Fig. 10 is an illustration in which the same focused 
region has been subjected to processing for super- 
imposing a predetermined image; 
Fig. 12 is an illustration of output data of the same 
data; 

Fig. 13 Is a block diagram of a communications con- 



trol unit in a second embodiment; 
Fig. 14 is an explanatory view of a meta-data de- 
scribing fonnat of the sam.e; 
Fig. 15 is an illustration describing the same meta- 
5 data in the MPEG-7 format; 

Fig. 1 6 is an explanatory view of a transmitting and 
receiving condition between the same communica- 
tions control unit and respective communications 
devices; 

10 Fig. 1 7 is an explanatory view of a group setting ex- 
ample of the same; 

Fig. 1 8 is an explanatory view of a transmitting and 
receiving condition between the same communica- 
tions control unit and respective communications 
IS devtees. 

[0023] Hereinafter, embodiments of the invention will 
be described with reference to the accompanying draw- 
ings. 

20 

(First embodiment) 

[0024] In the first embodiment, a case where one-on- 
one communications are carried out by a communica- 
^5 tions method and a device for the same according to the 
invention will be described. Fig. 1 is a block diagram of 
a communications device in the first embodiment of the 
invention. 

In this communications device, an input means 2, a dis- 
30 play means 3, and an image input means 4 face a user 
1 side. 

[0025] The input means 2 comprises a keyboard (in- 
cluding a numeric keypad, etc.) and a mouse, and is 
used by the user 1 to input control information on dis- 
ss closure or nondisclosure and other necessary informa- 
tion into this communications device. 
[0026] Herein, in a case where this communications 
device is used as a monitoring system or a surveillance 
system using the image input means 4, it is desirable 
40 that a reader is added to the input means 2. 

[0027] Thereby, even If the user 1 does not carry out 
keyboard input, control information on disclosure or 
nondisclosure of an image shot by the image input 
means 4 can be inputted into this communications de- 
45 vice. 

[0028] For example, the user 1 carries a batch with 
an IC card or an ID chip built-in wireless tag, which can 
carry out wireless communications with the input means 
2, and records control information on disclosure and 
50 nondisclosure onto this IC card or ID chip in advance. 
Thereby, the user 1 can easily input this control infor- 
mation into the communications device when carrying 
the batch or wireless tag. 

[0029] Disclosure control information indicates 
55 whether or not the user 1 will disclose either or both of 
a focused region and a non-focused region which in- 
cludes a part or all of an object, as described in detail 
later. 
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[0030] The display means 3 consists of an LCD or the 
like, and images transmitted from a communications 
partner and information necessary for operations are 
displayed on the display screen. 
[0031] The image input means 4 is used for inputting 
an image series consisting of at least one or more im- 
ages into this communications device, and various 
types exist such as a camera, a data input device, and 
a data recording medium. Normally, as this image se- 
ries, an image series imaging the user 1 is mainly used. 
[0032] A region extracting means 5 extracts a focused 
region including a part or all of an object from images of 
this image series. 

[0033] A focused region is a region including a part or 
all of an object, and a non-focused region is a region 
other than the focused region 1 in an image, for exam- 
ple, a background. 

[0034] Conventionally, various techniques have been 
disclosed as means for extracting an object region. 
[0035] For example, in Japanese Unexamined Patent 
Publication No. H05-91407, a relative comparison be- 
tween video signals of adjacent frames of a moving pic- 
ture is carried out, and an image portion in which the 
amount of action is small is defined as a background 
portion, and a portion other than the background portion 
is extracted as an object region. 

[0036] Furthermore, in Japanese Unexamined Patent 
Publication No. H05-161131 , an image including only a 
background is held in advance, and differences be- 
tween the image including only the background and re- 
spective frames of a moving picture are detected for 
each pixel, and a region in which such a difference is 
small is defined as a background region, and a region 
with a large difference is defined as an object region. 
[0037] Furthermore, in a case where the focused re- 
gion is a head or facial region, according to "Detection 
and tracing of a person's head based on an adaptable 
vision model using a fixed-viewpoint pan tilt zoom cam- 
era" by Taniuchi (pp. 9-14, Lecture Collected Papers I, 
Meeting on Image Recognition and Understanding 
(MIRU2000), an elliptic region in an image is detected 
to detect a head portion. 

[0038] In addition, there are generally-known various 
methods in which a face is detected based on color in- 
fomnation (for example, a skin-color region is detected), 
a facial portion such as the eyes or mouth is focused, 
or template-matching is used). 

[0039] Including the methods mentioned above, vari- 
ous techniques for region extraction have been dis- 
closed, and any of these techniques can be used in this 
region extracting means 5. 

[0040] Furthermore, in the invention, the purpose of 
region extraction is to protect privacy, so that it is not 
always necessary to accurately extract an object region, 
and even a manner In.which a part or all of an object is 
concealed to protect privacy Is also sufficient. For ex- 
ample, when a focused region is a head or facial region, 
an elliptic region including a part or all of the focused 



region may be detected. 

[0041] Furthenmore, in the present embodiment, the 
region extracting means 5 automatically extracts a fo- 
cused region, however, in a case where the location of 
5 the focused region in an image inputted from the image 
input means 4 is already known, the region extracting 
means 5 can be omitted. 

[0042] Furthermore, the invention targets privacy pro- 
tection, so that it is desirable that a portion showing per- 

10 son's characteristics, particularly the face, head, or a 
part thereof is defined as a focused region. 
[0043] Furthenmore, the user 1 or an administrator of 
this communications device may set a focused region 
as a fixed region previous to shooting, or the user 1 or 

15 an administrator of this communications device manu- 
ally sets a focused region while monitoring images shot 
by the image shot by the image input means 4. 
[0044] A transmitting data processing means 6 pre- 
pares control information in which an image inputted 

20 from the image input means 4, a focused region extract- 
ed from the region extracting means 5, and disclosure 
control infonnation inputted from the input means 2 are 
related to each other, and prepares data for communi- 
cations together with the image. 

25 [0045] Depending on a communications partner, 
transmitting data for which processing has been applied 
according to the focused region and the non-focused re- 
gion so that the details of a nondisclosure region are 
prevented from being revealed to the communications 

30 party is prepared. The details of this will be described 
later. 

[0046] A communicating means 7 transmits or re- 
ceives at least the abovementioned data to or from an 
opposite information processing device viacommunica- 
35 tions paths. Herein, a communications mode is optional, 
for example, synchronized communications as in the 
case with a videophone, or asynchronous communica- 
tions such as electronic mail. 

[0047] A receiving data processing means 8 process- 
40 es data received from a communications partner side 
via the communicating means 7 so that the data can be 
recorded into a recording means 9 or displayed on the 
display means 3. 

[0048] The recording means 9 consists of a memory 
45 or a hard disk device, and has a region for recording at 
least receiving data of the receiving data processing 
means 8. 

[0049] A control means 10 controls data flows and 
processing of the respective means shown in Fig. 1 . 

so [0050] Next, with reference to Fig. 2 through Fig. 5, 
an image series (including still images) to be communi- 
cated between communications devices and disclosure 
control infomnation in the communications method of the 
present embodiment will be described. 

55 [0051] An image series is, as mentioned above, data 
inputted from the image input means 4. An image series 
is generally tenned a type of contents, and it is desirable 
that disclosure control information is described in meta- 
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data concerning a contents description generated to be 
paired with an image series that is one piece of the con- 
tents. 

[0052] Fig. 2 shows a meta-data describing format to 
be used in this embodiment. In this description, "Region 
infomrjation" indicates which of a focused region and a 
non-focused region a target region is. 
[0053] "Disclosure control information" is a flag indi- 
cating whether or not a target region is to be disclosed. 
For example, at a certain stage of the contents, to define 
a focused region as a nondisclosure region, disclosure 
control information of region information on the focused 
region is set as "nondisclosure". 
[0054] In anotherf rame, to change the focused region 
to a disclosure region, the disclosure control information 
is changed to "disclose". This setting can be made for 
each of a focused region and a non-focused region. This 
setting is made by the user 1 via the input means 2. 
[0055] Furthermore, in place of the disclosure and 
nondisclosure flags, values of disclosure levels may be 
set. For example, when disclosure levels are indicated 
by values of 0 through 1 , the disclosure level "0" indi- 
cates nondisclosure, and the disclosure level "1" indi- 
cates disclosure. 

[0056] in a case of values between 0 and 1 , for exam- 
ple, a disclosure level "0.8", processing for varying the 
disclosure degree using parameter control in image 
processing is executed. 

[0057] "Region specifying information" is data show- 
ing a position of a target region in an image (for example, 
a contour coordinate row). 

[0058] There is a case where a target region Includes 
the whole Image and a case where a target region does 
not exist in an image. As a method for specifying a target 
region, in addition to setting a contour coordinate row, 
setting of binary data as mask data is possible. 
[0059] Furthermore, in a case where the shape (rec- 
tangular, polygon, or ellipse) of a target region is known 
in advance^ the position may be indicated by only infor- 
mation that is sufficient to show the position. The inven- 
tion can also be applied to not only one image but also 
a moving picture. 

[0060] In this case, as infomriation specifying a target 
region, not only information showing its spacial position 
but also infonmatfon showing time inten/als or infonna- 
tion showing the manner of spacial movements of a tar- 
get region may be used. These cases are also included 
in the invention. 

[0061] Furthermore, in addition to disclosure control 
information, descriptions concerning contents such as 
"contents encoding types" are Included in meta-data. 
Furthermore, the contents may be described as a table 
or hierarchically described. In the present embodiment, 
as shown in Fig. 2, disclosure control information is hi- 
erarchically described so as to be dependent on region 
information. 

[0062] Of course, it is also possible that region infor- 
mation and disclosure control inf omiation are independ- 



ently'described and the relationship between them is 
separately described. Any of the abovementioned cas- 
es is included in the invention only if disclosure control 
infomriation and region information are set in any fomi. 
5 [0063] Fig. 3 illustrates the relation of an image series 
and meta-data. In this example, an image series con- 
sists of a row of a plurality of still images (referred to as 
frames). 

[0064] In this image series, for example, an optional 
10 appropriate fonnat such as an analog, digital, interlace, 
progressive, encoding, or decoding fomnat can be used. 
[0065] Among the frames, there are four frames in- 
cluding an object (in this example, a face of a young 
woman) and two frames that do not include it. 
[0066] Disclosure control infonnation to be described 
in meta-data can be related to several optional frames 
In an Image series such as one frame, a sequential 
frame group, a non-sequential frame group, or the entire 
image series. 

[0067] For the disclosure control information, a flag 
indicating whether or not region information is to be dis- 
closed is set. For example, at a certain stage of the con- 
tents, in a case where a focused region is set as a non- 
disclosure region, a flag is set as "nondisclosure". 
[0068] Furthermore, in a different frame, in a case 
where a focused region is changed to allow disclosure, 
at this point, a flag "disclosure" is set. 
[0069] As a format describing meta-data used herein, 
various standards and unique standards exist. Herein, 
a description giving an example based on the MPEG-7 
will be described. 

[0070] The MPEG-7 is the fourth standardizing Item 
following the MPEG-1 , MPEG-2, and MPEG-4 of MPEG 
(Moving Picture Experts Group: ISO/I EC 
JTG1/SC29/WG11) fomially called Multimedia Content 
Description Interface (ISO/IEC 15938). 
[0071] This standardization includes standards to be 
used for regulating a schema for describing the contents 
of multimedia information and used for applications of 
retrieval and editing from a digital library. 
[0072] Furthermore, the MPEG-7 regulates a stand- 
ard group of descriptors for describing the contents of 
multimedia Information mainly corresponding to images 
and sound. By providing description using descriptors 
for the contents, retrieval based on the contents of the 
multimedia information becomes possible. 
[0073] This is intended to promote circulation and use 
of the contents by this. Therefore, a description fomn ac- 
cording to said standards is preferable in the invention 
that has been developed for the purpose of communi- 
cations involving privacy protection. 
[0074] As an actual description definition language, a 
language obtained by applying necessary extension to 
the XML (extensible Markup Language) Schema lan- 
guage is used. Furthemiore, compatibility with the 
grammar of the XML Schema is maintained even after 
such extension. 

[0075] In the MPEG-7, to describe a characteristic ex- 
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pression of the contents, the following basic elements 
may be combined. 

1 . Descriptor {abbreviated to D) 

[0076] This is a basic tool for describing a single char- 
acteristic of multimedia contents. In the MPEG-7, a no- 
tation system (Syntax) and meaning (Semantics) of de- 
scriptors are regulated. 

2. Description schema (abbreviated to DS) 

[0077] This schema regulates the structures and se- 
mantic relationships among a plurality of describing 
tools. As descriptors, in the MPEG-7, a notation system 
(Syntax) and meanings (Semantics) of the description 
schema are regulated. In describing tools composing 
the description schema, other description schemata 
themselves are also included in addition to the descrip- 
tors. 

3. Description definition language 

[0078] This is a language for regulating a notation sys- 
tem of the descriptors and the description schema. In 
the MPEG-7, by usingthe XML schema that is aschema 
language standardized and fomnulated according to 
W3C as a base, various data types that become neces- 
sary to describe characteristics of multimedia contents 
are added to regulate the description definition lan- 
guage. 

[0079] As examples of DS, "Person DS" for describing 
contents users, "Still Region DS" for describing regions 
within an image, and "Usage Infomnation DS" for de- 
scribing a method for using contents exist. 
[0080] As an example of a descriptor, "Rights" for de- 
scribing an access right to contents exists. 
[0081] In the MPEG-7, in addition to the DS and de- 
scriptor defined as standards, new descriptors and lan- 
guages for defining and extending the DS (Description 
definition language) are regulated. 
[0082] Therefore, a description of meta-data in the in- 
vention becomes a description in the MPEG-7 format by 
description according to the abovementioned regula- 
tions. 

[0083] Fig. 4 shows an example of a description of the 
meta-data shown in Fig. 3 according to the MPEG-7 
standards. In Fig. 4, "<!- -" is a comment, and region 
information is described by using "Still Region DS". 
[0084] Disclosure control infonmation is described by 
"Publication Type" In "Availability DS" of "Usage Infor- 
mation DS". 

[0085] In this "Publication Type", a mechanism for dis- 
tribution of these contents can be described. 
[0086] Herein, by describing disclosure/nondisclo- 
sure information that the disclosure control mechanism 
uniquely provides, a mechanism of disclosure control in 
a case of distribution of contents is detemiined. 



[0087] Region specifying information is described as 
a contour coordinate series by using "Contour Shape". 
[0088] As mentioned above, as region information, in 
addition to region information of one image, information 
5 of a moving region "Moving Region DS" and region in- 
formation "Audio Visual Region DS" integrated with 
sounds exist. 

[0089] As a basic definition including these, there is 
"Segment DS" showing a part of multimedia contents, 
10 and according to a DS based on this definition, a de- 
scription equivalent to the description shown in Fig. 4 is 
possible. 

[0090] Next, relevance to a moving picture will be de- 
scribed in detail. Fig. 5 shows an example of a descrip- 
15 tion of the meta-data of Fig. 3 according to MPEG-7 
standards. 

[0091] In Fig. 5, the region information shown in Fig. 
4 is related to a target image frame. In this case, actual 
relation to an image frame is carried out in "Media Time 

20 Point" of "Media Time". 

[0092] In "Media Time", a target video segment time 
is described in the format shown in Fig. 6. For example, 
'T1 3:20:01 :1 F15" means "the first frame at thirteen hun- 
dred hours twenty minutes and one second (15 frames 

25 per minutes, 0 through 14 frames provided)". 

[0093] Thereby, it becomes possible to control only a 
partial section in a moving picture so as to be disclosed. 
[0094] The abovementioned description is just an ex- 
ample, and other description methods are included in 

30 the invention if an equivalent meaning is obtained. 
[0095] In the M PEG-7, meta-data Is expressed by the 
XML. Currently, XML has received significant attention 
in the field of infomnation processing due to its versatility, 
and various tools for editing and operating XML docu- 

35 ments have been provided, and since MPEG-7 meta- 
data is expressed by using XML, it can receive maxi- 
mum benefits of these XML versatile tools. 
[0096] Furthermore, affinity with other standards 
based on XML, such as HTML (Hyper Text Markup Lan- 

40 guage) and SMIL (Synchronized Multimedia Integration 
Language) is high. 

[0097] In the invention, a description is sufficient if the 
description is made in a format related to contents, so 
that various meta-data descriptions are possible includ- 

45 ing the abovementioned standards. 

[0098] In the invention , a meta-data description is ex- 
plained based on a description in a text format for con- 
venience of explanation in the drawings, however, a bi- 
nary format can also be used, and is included in the in- 

50 vention. 

[0099] For example, in a case where various charac- 
teristics of multimedia contents are described as an 
XML document, there is a concern that the size of the 
MPEG-7 meta-data itself becomes massive, resulting in 
55 storage or transmission inefficiency. 

[0100] As a countermeasure for this, use of a binary 
format regulated in the MPEG-7 is considered. Thereby, 
it becomes possible to compress the MPEG-7 meta-da- 
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ta described as an XML document up to a size of 1/1 00 
or less without a loss of the information contents. In this 
format, a meta^data description according to the text for- 
mat and a meta-data description according to the binary 
fomriat are equivalent to each other as information of 
these pieces of data. 

[0101] Therefore, there is bidirectional ity between the 
text format and the binary format, and even when 
MPEG-7 meta-data described as an XML document is 
temporarily converted into MPEG-7 binary data and 
then converted into XML text data again, the original in- 
fonnation is perfectly maintained. 
[0102] Even in a binary format, random access to the 
data contents is possible, and in addition, operations 
such as deletion of only a part of a description inside 
data or rewriting of other contents are also possible. 
[0103] Herein, in the invention, a description is suffi- 
cient if the description is made as meta-data, and de- 
scriptions in any of the abovementioned formats are al- 
so included in the invention. 

[0104] Next, with reference to Fig, 7 through Fig. 12, 
processing to be executed by the transmitting data 
processing means 6 will be described. 
[0105] This transmitting data processing means 6 
prepares control information In which an Image Inputted 
from the image Input means 4, a focused region extract- 
ed by the region extracting means 5, and disclosure con- 
trol info mnat ion inputted from the input means 2 are re- 
lated to each other. 

[0106] Herein, in this example, the transmitting data 
processing means 6 executes processing so that other 
users are prevented from understanding the details of 
a region that is set as a nondisclosure region based on 
the control information. However, In place of the trans- 
mitting data processing means 6, this processing can 
be executed by a receiving data processing means 8 of 
the communications partner side which receives this 
control information and image, or by an information 
processing device on another communications path. 
[0107] in any case, it is only required that processing 
is executed by a certain means on a communications 
path until another user finally looks at the image so that 
the nondisclosure request is satisfied, and in the inven- 
tion, the processing manner is not limited to this case 
where the transmitting data processing means 6 exe- 
cutes the processing. 

[0108] Fig. 7 shows original Input data before being 
processed, and Figs. 8 through Fig. 11 show examples 
of images for which various processings have been ap- 
plied. These examples can be used by being optionally 
combined. 

[0109] Herein, the region extracting means 5 extracts 
the face as a focused region. 

[0110] Fig. 8 shows the results oi the processings to 
set the non-focused region (background) as a nondis- 
closure region. In order to set the background so as not 
to be disclosed, the background is shaded off in Fig. 8 
(a), and in Fig. 8(b), the background is darkened. 



[0111] On the contrary, Figs. 9 through Fig. 11 show 
examples of the results of the processings to set the fo- 
cused region (facial region) so as not to be disclosed. 
In Fig. 9(a), the focused region is shaded off, and in Fig. 

5 9(b), the focused region is mosaicked, and in Fig. 9(c), 
the focused region is embossed. 
[01 12] Fig. 1 0 shows an example in which, on the fa- 
cial region that is the focused region, a predetermined 
computer graphics (CG) character or character image 

10 that is different from the image of the focused region is 
superimposed. Furthermore, as shown In Fig. 11, it is 
also considered that a CG or an image representing 
sunglasses is superimposed on the eye portions in the 
facial region that is the focused region. 

'5 [0113] In addition to these processings, various 
processing methods can be considered, and the 
processings can be used only If they make it Impossible 
for other users to understand the details of the region 
set as a nondisclosure region in an image. Furthermore, 

20 the illustrated processings themselves can be carried 
out according to generally-known techniques. 
[01 14] Thus, in the present embodiment, the transmit- 
ting data processing means 6 executes processing 
which makes it impossible for other users to understand 

^5 the details of a nondisclosure region in an image. Fig. 
1 2 is an example of output data after applying process- 
ing to the input data of Fig. 3. 

[0115] As mentioned above, according to the inven- 
tion, an information processing device at a communica- 

30 tions partner side, which receives output data, acquires 
a moving picture in which a region set so as not to be 
disclosed has been processed, and meta-data including 
disclosure control information existing by being paired 
with the region. 

55 [0116] The abovementioned processing may be en- 
coding. In this case, 

[0117] (Example 1) a nondisclosure region is encod- 
ed by a public key, a public key certification is attached 
to the meta-data and then the data is transmitted so that 
a receiver who is pemnitted to look at this nondisclosure 
region can decode the nondisclosure region by a private 
key con^esponding to the public key, or (Example 2) a 
nondisclosure region is encoded by a common key, and 
the common key is encoded and attached to the meta- 

45 data, and then the data is transmitted so that a receiver 
who is permitted to look at this nondisclosure region can 
decode the common key by a private key to acquire the 
common key and decode the nondisclosure region by 
this common key. 

so [01 18] In (Example 1), there is an advantage that dif- 
ferent encodings can be easily applied to each nondis- 
closure region, and in (Example 2), there Is an advan- 
tage that the processing burden is lighter. 
[0119] Furthermore, in both of (Example 1), and (Ex- 

55 ample 2), a person who has a private key can decode 
and look at a nondisclosure region, and a person who 
does not have the key cannot look at the region. Namely, 
only specific persons can look at a nondisclosure region 
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of an image although the image is transmitted through 
the same network to the specific persons and others. 
[0120] The communications manner between the 
communications device of the present embodiment and 
a receiving information processing device may be syn- 
chronous communications such as videophones or 
asynchronous communications such as electronic mail. 
[0121] In any communications manner, there is a case 
where transmitting data transmitted by the communica- 
tions device of the present embodiment is recorded in 
a receiving side Infomriation processing device. 
[01 22] I n such a case, an image series and a descrip- 
tlon related to this are simultaneously received and re- 
corded, and therefore, even when it is desired later to 
reproduce only a disclosed portion, it is not necessary 
to search through the entire image series by using fast- 
forwarding, and it Is sufficient to search through contents 
describing data, and this makes data handling easier 
[0123] Therefore, according to the present embodi- 
ment, very easy and efficient retrieval can be carried out 
at high speed. 

[0124] Furthemiore, due to the abovementioned 
stream system, management of images themselves is 
also easy. For example, meta-data is allowed to include 
a communications date or time other than the disclosure 
control infomriation. Taking-in of such meta-data togeth- 
er with an image eliminates the possibility that Informa- 
tion such as date and time described in the meta-data 
and disclosure control information are lost, misplaced, 
or deleted by mistake. 

[01 25] Therefore, when image recording Is carried out 
by a recording means of a receiving side information 
processing device, it is not necessary to store infomria- 
tion on contents descriptions in a separate file, and the 
infomriation on contents descriptions can be managed 
by its file itself. Thereby, it is not necessary for a user to 
understand contents descriptions in advance, and a us- 
er can acquire image data on demand. 
[01 26] As mentioned above, according to the commu- 
nications method relating to the first embodiment of the 
invention, meta-data related to an image series exists, 
and disclosure control information is included in the me- 
ta-data. Therefore, total data management becomes 
easier when recording this data, and in a case where 
the contents are searched through afterwards, only a 
disclosed portion can be retrieved by using the Infomria- 
tion of the meta-data, so that data can be extremely ef- 
ficiently retrieved. 

[0127] In the present embodiment, an example of in- 
teractive communications by the communicating means 
7 has been explained, and furthermore, the present in- 
vention can be applied to video mail in which simplex 
communications of image data is carried out from the 
user 1 to a communications partner. 
[0128] In this case, an information processing device 
of the communications partner may only be constructed 
so as to receive and display transmitted image data. 



(Second embodiment) 

[0129] Next, as a second embodiment, a method is 
disclosed which Is effective in a case where the commu- 

5 nications partners which the communications device of 
the invention communicates with are 2 or more commu- 
nications devices, and transmitting data is to be dis- 
closed to a part of the communications partners, and is 
not to be disclosed to other parties. 

10 [0130] In this case, between a transmitting side com- 
munications device and a receiving side communica- 
tions device, the communications control unit of the in- 
vention, which controls communications between these 
devices is interposed. This communications control unit 

15 is also included in the "communications device" of the 
invention. 

[0131] Fig. 13 is a block diagram of the communica- 
tions control unit which realizes the communications 
method of the second embodiment of the invention. 
20 [0132] Herein, as shown in Fig. 16, communications 
among three communications devices A, B, and C are 
controlled by the communk;ations control unit 25 shown 
in Fig. 13. 

[0133] As shown in Fig. 13. the communications con- 
25 trol unit 25 comprises a communicating means 1 6, a re- 
ceiving data processing means 1 7. a recording means 
18, a transmitting data processing means 1 9, and acon- 
trol means 20. 

[0134] Herein, as shown'in Fig. 16, the communicat- 
ee ing means 1 6 exchanges data with the three communi- 
cations devices A, B, and C through communications 
paths. The communications mode herein is optional, for 
example, synchronous communications as in the case 
with videophones or asynchronous communications as 
35 in the case with electronic mail. 

[0135] The network configuration of Fig. 18 is also 
employed as well as that of Fig. 16. In this case, the 
communications devices B and C exist on the same net- 
work, and the communications device A can be connect- 
40 ed to this network only under a condition where the com- 
munications control unit 25 is interposed between the 
device and the network. 

[0136] Thereby, a raw image (including a nondisclo- 
sure region that has not been protected) picking-up the 

45 user A is transmitted to the communications control unit 
25 by the communications devfee A, but Is prevented 
from passing through this network unless some 
processing is applied to this raw image, and this im- 
proves confidentiality of the nondisclosure region. 

50 [0137] Therefore, such a network configuration (par- 
ticulariy, configuration between the communications de- 
vice A and the communications control unit 25) is useful 
for a service business internal network providing a video 
image check service using an IP camera or a moving 

55 picture distribution service. 

[0138] In Fig. 13, the receiving data processing 
means 1 7 receives data from the communications de- 
vices A, B, and C, and analyzes the received meta-data. 



8 



BNSDOCID: <EP. 



.1311124A1_L> 



15 

and acquires Information necessary for the transnnlttlng 
data processing means 19. 

[0139] This data includes, besides the control infor- 
nriatlon on disclosure and nondisclosure, at least one 
piece of distinguishing information such as an index for 
distinguishing users who use the respective devices A, 
B, and C, user groups to which the users belong, or de- 
vices which the users use. This data will be described 
in detail later. 

[0140] The recording means 1 8 stores all or a part of 
infonnation on users who use the communications con- 
trol unit, user groups to which the users belong, or de- 
vices which the users use. Furthemr^ore, this recording 
means 18 records data as the occasion demands when 
receiving the data. 

[0141] When an image series of the data received by 
the receiving data processing means 17 needs to be 
processed, the transmitting data processing means 19 
applies processing to the image series so that a nondis- 
closure region is prevented from being revealed. 
[01 42] Next, with reference to Fig. 1 4. a format of data 
to be communicated between communications devices 
according to the communications method of the present 
embodiment will be described. 

[0143] In Fig. 14, a meta-data describing format to be 
used in this embodiment is shown. In this embodiment, 
infonnation on a specific person (specific person infor- 
mation is newly added to the first embodiment. 
[0144] "Specific person" indicates a user for whom it 
is desired to set a value different from a value for un- 
specified persons for disclosure and nondisclosure of a 
region, for example, information on users, user group 
infonnation. affiliated organization infonnation, address 
infonnation including electronic addresses, and terminal 
distinguishing information. 

[0145] Herein, in the MPEG-7, "agent" is the general 
term for objects which handle data to be communicated. 
As shown in Fig, 14, region information includes disclo- 
sure control Information for specific persons separately 
from disclosure control information for unspecified per- 
sons. 

[0146] The contents of disclosure control information 

are the same as in the first embodiment. 
[0147] Thus, by correcting the format of the first em- 
bodiment into the format of Fig. 14, for example, in a 
case where data is transmitted from the communica- 
tions device A to the communications devices B and C, 
a user of the communications device B is set as a "spe- 
cific person" and handled differently from unspecified 
persons (users of the communications devices A and 
C), whereby control between disclosure and nondisclo- 
sure can be made. 

[0148] Fig. 15 shows an example of a description by 
using the MPEG-7 format for the meta-data describing 
format of Fig. 14. Herein, in comparison with Fig. 4 re- 
lating to the first embodiment, "Person Group DS" is 
newly and additionally used as specif ic person informa- 
tion. 
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[0149] For this Person Group DS, a unique name in- 
dicating, for example, a user's friends group can be pro- 
vided as specific person information, 
[0150] In this embodiment, an example in which a 

5 group's unique name has been provided in advance Is 
described, however, it is also possible that the user's 
friends group is defined in advance. In this case, it is 
defined in Person Group DS as to what members are to 
be included in the group. 

10 [0151] As specific person infonnation, Person Group 
DS has been described herein, however, other than this, 
in the MPEG-7, there are "Person DS" indicating per- 
sons. "Organization DS" indicating organizations, 
"Agent DS" as the general tenn defining objects that 

15 handle data to be communicated, and "Place DS" indi- 
cating places. All of these have an effect equivalent to 
that of the example of Person Group DS and is Included 
in the invention only If It is Infonnation that specifies us- 
ers. 

20 [0152] Next, an explanation is given about a flow of 
control of disclosure/nondisclosure of an image series 
depending on the communications partners A, B, and C 
by the communications control unit 25 based on the spe- 
cific person information in the abovementioned meta- 

^5 data description. 

[0153] Fig. 16 is a schematic diagram showing com- 
munications among at least three communications de- 
vices A, B, and C via the communications control unit 25. 
[0154] Herein, it is assumed that data is transmitted 

30 from the communications device A to the communica- 
tions devices B and C, and the facial region is set as a 
focused region. 

[0155] In the meta-data to be transmitted from the 
communications device A, user B distinguishing infor- 
ms mation has been set as specific person information. 
[0156] In this case, when data is transmitted from the 
communications device A to the communications devic- 
es B and C via the communications control unit 25, 
transmitting data is transmitted in the same form as the 
40 data 29 in which the focused region has been disclosed. 
[01 57] Then, the receiving data processing means 1 7 
of the communications control unit 25 (see Fig. 13) ac- 
quires distinguishing information and disclosure control 
information from the meta-data, and in a case where the 
45 transmission destination is the user B, according to the 
disclosure control infonnation for a specific person, the 
transmitting data processing means 1 9 transmits the da- 
ta in the form of data 30 in which the focused region has 
been disclosed. On the other hand, to the user C who 
50 is not a specific person, the transmitting data processing 
means 19 applies processing to conceal details of the 
facial region that is a nondisclosure region in the same 
manner as data 31 and then transmits the processed 
data. 

55 [01 58] For this specific person information, not a user 
but a user group can be set. This user group is like a 
mailing list in nonnal electronic mall systems, and this 
information Is set in the recording means 1 8 of the com- 
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according to the control infornnation so that the de- 
tails of the region are prevented fronn being re- 
vealed. 

5 3. The connmunications method using images accord- 
ing to Claim 1 , wherein the image series is an en- 
coded moving picture. 

4. The communications method using images accord- 
10 ing to Claim 2, wherein said processing is image 

processing for lowering the image quality of a non- 
disclosure region. 

5. The communications method using images accord- 
15 ing to Claim 4, wherein said processing is process- 
ing for lowering the resolution of a nondisclosure re- 
gion. 

6. The communications method using images accord- 
20 ing to Claim 4, wherein said processing is process- 
ing for eliminating a nondisclosure region orfor con- 
verting expression of a nondisclosure region using 
binarization processing. 

25 7. The communications method using images accord- 
ing to Claim 2, wherein said processing is process- 
ing for encoding a nondisclosure region. 

8. The communications method using images accord- 
30 Ing to Claim 4, wherein the images compose an en- 
coded moving picture, and said processing is 
processing for controlling encoding parameters 
when carrying out this encoding so that disclosure 
and nondisclosure regions are correctly disclosed 
35 and not disclosed. 



munications control unit 25. Fig. 17 shows an example 
of such a group setting. 

[0159] In this case, group information is set as a list 
structure divided for each group as shown in Fig. 17, 
and other than this, a list configuration in which informa- 
tion for distinguishing which of the groups the users be- 
long to is added to the user list may be employed. 
[0160] Furthermore, a method may be employed in 
which infonnation on the group and organization is ex- 
changed during communications with the respective 
communications destinations and midstream results in- 
formation Is stored in the recording means 18. 
[0161] All of these methods are included in the inven- 
tion on condition that the communications control unit 
25 can recognize the user group distinguishing infornna- 
tion. 

[01 62] As described above , according to the commu- 
nications method of the embodiment, an image can be 
transmitted in different manners of region disclosure to 
specific persons and unspecified persons. 
[01 63] Therefore, when a user desires not to disclose 
a part of an image to a specific person for privacy pro- 
tection, the part can be set so as not to be disclosed, 
and when the user desires to disclose the part to only a 
specific person, the part can be set so as to be dis- 
closed. This communications method is very convenient 
for users, in which privacy protection is taken into ac- 
count, 

[0164] As described above, according to the inven- 
tion, image communications can be carried out while se- 
curing privacy protection. 

[0165] Having described preferred embodiments of 
the invention with reference to the accompanying draw- 
ings, it is to be understood that the invention is not lim- 
ited to those precise embodiments, and that various 
changes and modifications may be effected therein by 
one skilled in the art without departing from the scope 
or spirit of the invention as defined in the appended 
claims. 



Claims 

1. A communications method using images, which 
communicates an Image series including one or 
more images and control Infonnation that is related 
to at least one of the images of the image series and 
concerns disclosure/nondisclosure of the image, 
wherein 

this control information is related to either or both of 
a focused region and a non-focused region which 
include a part or all of an object within the image, 
and includes information as to whether or not the 
related region is to be disclosed. 

2. The communications method using images accord- 
ing to Claim 1 , wherein processing is applied to a 
region that has been set as a nondisclosure region 



9. The communications method using images accord- 
ing to Claim 1 , wherein the focused region is a skin- 
color region. 

40 

10. The communications method using images accord- 
ing to Claim 1 , wherein the focused region is a re- 
gion including a part or all of the face and head. 

45 11. The communications method using images accord- 
ing to Claim 1 , wherein a nondisclosure region is a 
region including a part or all of the face and head, 
and said processing is processing for superimpos- 
ing an object so as to cover a part or all of the non- 
50 disclosure region. 

12. The communications method using images which 
communicates an image series including one or 
more images and control information that is related 
55 to at least one of the images of the image series and 
concerns disclosure/nondisclosure of the image, 
wherein 

said control information includes infornnation on dis- 
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closure or nondisclosure to specific users separate- 
ly fronn unspecified users. 

13. The communications method using images accord- 
ing to Claim 12, wherein Infomnation on disclosure 
or nondisclosure to specific users is composed of 
at least one kind or a combination of two or more 
kinds among information on users who use the im- 
age series, user group information, affiliated organ- 
ization information, address infonnation including 
electronic addresses, terminal distinguishing infor- 
mation, and agent information on agents which ac- 
cess the Images. 

14. The communications method using images accord- 
ing to Claim 12, wherein information on disclosure 
or nondisclosure to specific users includes any in- 
formation of a key which has encoded a nondisclo- 
sure region, information of said key that has been 
encoded, certification information on the key that 
has encoded the nondisclosure region, and infor- 
mation of said certification information that has 
been encoded. 

15. The communications method using images accord- 
ing to Claim 1 , wherein, in a case where an image 
is communicated to a specific user, disclosure of the 
related region is controlled according to the control 
information on disclosure/nondisclosure to this spe- 
cific user. 

16. The communications method using images accord- 
ing to Claim 1 , wherein control information is data 
described in the form of meta-data related to at least 
one of the images of the image series. 

17. The communications method using images accord- 
ing to Claim 16, wherein the meta-data is described 
in a standardized fomnat. 

18. The communications method using images accord- 
ing to Claim 1 7, wherein the standardized format is 
the MPEG-7 format. 

19. A communications device comprising: 

communicating means(7) operable to commu- 
nicate data with opposite communications de- 
vices via communications paths; 
recording means(9) operable to recorde an Im- 
age series including one or more images; 
input means(2) operable to input disclosure 
control infonnation indicating whether or not a 
specific region forming a part or ail of an image 
is to be disclosed to the communications part- 
ners; and 

transmitting data processing means(6) opera- 
ble to prepare transmitting data to which 



processing has been applied according to dis- 
closure control Information so as to prevent de- 
tails of a nondisclosure region in an image from 
being revealed to the communications partners 
5 and transmitting this transmitting data to the 

communications devices of the communica- 
tions partners by using said communicating 
means(7). 

10 20. The communications device according to Claim 19, 
wherein said disclosure control information consists 
of only disclosure control information for unspeci- 
fied users. 

'5 21. The communications device according to Claim 20, 
wherein said disclosure control information in- 
cludes disclosure control infomriation for specific us- 
ers. 

20 22. The communications device according to Claim 19, 
wherein said disclosure control infomnation is de- 
scribed as meta-data in the MPEG-7 format. 
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Fig. 3 
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Fig. 4 
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! — ###########ff##M#####ff#### — ^ 

<! — example of a description of the meta-data in Fig. 3 — > 

<!— #################################^ ~> 

<! — region information — > 
<StillRegion id="target region"> 

<! — (A) disclosure control information — > 

<Usage Information id="disclosure control information"> 

<Avai labi I ity id="disclosure control for unspecified persons"> 
<Publication Type href="disclosure control meohanism"> 

<Name>d i sc I osure</Name> 
</Pub 1 i cat i on Type> 
</Avai labi I ity> 
</Usage I nf ormat i on> 

<! — (B) region specifying information — > 
<Contou r Shape number Of Peaks=''54"> 

<GlobalCurvatureVecter>3803. . . 12215</6lobalCurvatureVector> 
<PrototypeCurvatureVector>3502. . . 7500</PrototypeCurvatureVector> 
<H i ghestPeakM 5686</H i ghestPeak> 

<Peak><xpeak>20862</xpeak><ypeak>14886</ypeak></Peak> 
<Peak><xpeak>34229</xpeak><ypeak>7442</ypeak></Peak> 
<Peak><xpeak>51 741 </xpeak><ypeak>5485</ypeak></Peak> 

</ContourShape> 

</StillRegion> 
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Fig. 5 

<!~ ############################################################ ~> 

<! — example of a description of the meta-data in Fig. 3 — > 

<!— ########|f################# M MMM m — > 

,.4 

<VideoSegment id="lst contror'> 
<MediaTime> 

<Med i aT imePo i nt>Tl 3 : 20 : 01 : 1 F1 5</Med i aT i mePo i nt> 

</MediaTime> 

<! — 1st region information — > 
<StillRegion id="target region"> 

</StillRegion> 
</V i deoSegment> 

<VideoSegment id="2nd contror'> 
<MediaTime> 

<Med i aT 1 mePo i nt>T 1 3 : 23 : 20 : 1 5F1 5</Med i aT 1 mePo i nt> 
</MediaTime> 

<! — 2nd region information — > 
<StillRegion id="target region"> 

</StillRegion> 
</V i deoSegment> 

<VideoSegment ld="2nd control"> 
<MediaTime> 

<Med i aT imePo i nt>T1 3 : 29 : 02 : 3F1 5</Med i aT imePo i nt> 
</MediaTime> 
<! — 2nd region — ^> 
<StillRegion id="target region"> 

</StiMRegion> 
</V i deoSegment> 



15 



BNSOCX:iD: <EP 1311124A1_L> 



EP 1 311 124 A1 



Fig. 6 
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Fig. 13 
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Fig. 14 
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Fig, 15 

< I — ###tf#Mffff#fffftfffffff#ff#ff#fffffffW — > 

^ • Trrr If n tt TtTttrTfTrtfTrTrTrtrtrTf trTrfrtrff I 'tTrfrTrTrTrfr / 

<! — region disclosure control (many points) — > 

<! — specific person information — > 
<PersonGroup> 

<Naine Type="var iant">My Friends' Unique Name</Name> 
</PersonGroup> 

<! — region informat ion ( including disclosure control information) — > 
<StillRegion id="UserFaseSR"> 

<! — disclosure control information — > 

<Usage I nf ormat i on id="disclosure control informat ion"> 

<! — disclosure control information for specific persons — > 
<Avai lability id="di sc losure control for specific persons"> 
<Pub Meat ion Type href="disclosure control mechanism"> 

<Name>d i sc I osure</Name> 
</Publ icat ion Type> 
</Avai labi I ity> 

<! — disclosure control information for unspecified persons — > 
<Avai labi I ity id="disclosure control for unspecified persons"> 
<Publication Type href="disclosure control mechanism"> 

<Name>nond i sc I osur e</Name> 
</Publ ication Type> 
</Availability> 
</Usage I nf ormat i on> 

<! — region specifying information — > 
<ContourShape number0fPeaks=="54"> 

<GlobalCurvatureVecter>3803. . . 12215</GlobalCurvatureVeGtor> 
<PrototypeCurvatureVector>3502. . . 750G</PrototypeCurvatureVector> 
<H i ghestPeak>1 5686</H i ghestPeak> 

<Peak><xpeak>20862</xpeak><ypeak>14886</ypeak></Peak> 
<Peak><xpeak>34229</xpeak><ypeak>7442</ypeak></Peak> 
<Peak><xpeak>51 741 </xpeak><ypeak>5485</ypeak></Peak> 

</ContourShape> 

</StiriRegion> 
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Fig. 18 
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